XCS with eligibility traces

Belgium Herbarium image of Meise Botanic Garden

The Cost of Accumulating Evidence in Perceptual Decision Making

Decision making often involves the accumulation of information over time, but acquiring information typically comes at a cost. Little is known about the cost incurred by animals and humans for acquiring additional information from sensory variables due, for instance, to attentional efforts. Through a novel integration of diffusion models and dynamic programming, we were able to estimate the cost of making additional observations per unit of time from two monkeys and six humans in a reaction time (RT) random-dot motion discrimination task. Surprisingly, we find that the cost is neither zero nor constant over time, but for the animals and humans features a brief period in which it is constant but increases thereafter. In addition, we show that our theory accurately matches the observed reaction time distributions for each stimulus condition, the time-dependent choice accuracy both conditional on stimulus strength and independent of it, and choice accuracy and mean reaction times as a function of stimulus strength. The theory also correctly predicts that urgency signals in the brain should be independent of the difficulty, or stimulus strength, at each trial

Speed and Accuracy of Static Image Discrimination by Rats

When discriminating dynamic noisy sensory signals, human and primate subjects achieve higher accuracy when they take more time to decide, an effect attributed to accumulation of evidence over time to overcome neural noise. We measured the speed and accuracy of twelve freely behaving rats discriminating static, high contrast photographs of real-world objects for water reward in a self-paced task. Response latency was longer in correct trials compared to error trials. Discrimination accuracy increased with response latency over the range of 500-1200ms. We used morphs between previously learned images to vary the image similarity parametrically, and thereby modulate task difficulty from ceiling to chance. Over this range we find that rats take more time before responding in trials with more similar stimuli. We conclude that rats' perceptual decisions improve with time even in the absence of temporal information in the stimulus, and that rats modulate speed in response to discrimination difficulty to balance speed and accuracy

Lateral orbitofrontal cortex anticipates choices and integrates prior with current information

Adaptive behavior requires integrating prior with current information to anticipate upcoming events. Brain structures related to this computation should bring relevant signals from the recent past into the present. Here we report that rats can integrate the most recent prior information with sensory information, thereby improving behavior on a perceptual decision-making task with outcome-dependent past trial history. We find that anticipatory signals in the orbitofrontal cortex about upcoming choice increase over time and are even present before stimulus onset. These neuronal signals also represent the stimulus and relevant second-order combinations of past state variables. The encoding of choice, stimulus and second-order past state variables resides, up to movement onset, in overlapping populations. The neuronal representation of choice before stimulus onset and its build-up once the stimulus is presented suggest that orbitofrontal cortex plays a role in transforming immediate prior and stimulus information into choices using a compact state-space representation

Market Imitation and Win-Stay Lose-Shift Strategies Emerge as Unintended Patterns in Market Direction Guesses.

Decisions made in our everyday lives are based on a wide variety of information so it is generally very difficult to assess what are the strategies that guide us. Stock market provides a rich environment to study how people make decisions since responding to market uncertainty needs a constant update of these strategies. For this purpose, we run a lab-in-the-field experiment where volunteers are given a controlled set of financial information -based on real data from worldwide financial indices- and they are required to guess whether the market price would go "up" or "down" in each situation. From the data collected we explore basic statistical traits, behavioural biases and emerging strategies. In particular, we detect unintended patterns of behavior through consistent actions, which can be interpreted as Market Imitation and Win-Stay Lose-Shift emerging strategies, with Market Imitation being the most dominant. We also observe that these strategies are affected by external factors: the expert advice, the lack of information or an information overload reinforce the use of these intuitive strategies, while the probability to follow them significantly decreases when subjects spends more time to make a decision. The cohort analysis shows that women and children are more prone to use such strategies although their performance is not undermined. Our results are of interest for better handling clients expectations of trading companies, to avoid behavioural anomalies in financial analysts decisions and to improve not only the design of markets but also the trading digital interfaces where information is set down. Strategies and behavioural biases observed can also be translated into new agent based modelling or stochastic price dynamics to better understand financial bubbles or the effects of asymmetric risk perception to price drops